Linear reinforcement learning in planning, grid fields, and cognitive control

Abstract It is thought that the brain’s judicious reuse of previous computation underlies our ability to plan flexibly, but also inappropriate gives rise inflexibilities like habits and compulsion. Yet we lack a complete, realistic account either. Building on control engineering, here introduce model for decision making in brain reuses temporally abstracted map future events enable biologically-realistic, flexible choice at expense specific, quantifiable biases. replaces classic nonlinear, model-based optimization with linear approximation softly maximizes around (and weakly biased toward) default policy. This solution demonstrates connections between seemingly disparate phenomena across behavioral neuroscience, notably replanning biases cognitive control. provides insight into how can represent maps long-distance contingencies stably componentially, as entorhinal response fields, exploit them guide even under changing goals.